Multiple simultaneous audio video data decoding

ABSTRACT

A system includes a server configured to receive a data stream and at least one client device configured to receive the data stream from the server. The server is configured to synchronize playback of a media content instance by the server, one or more of the client devices, or both, in accordance with a timestamp associated with the data stream.

BACKGROUND

Networked televisions are gaining popularity in homes and businesses.Consumers want to watch recorded shows, live shows, or both, fromdifferent locations. In a residential setting, consumers will beginwatching a show in one location, such as a family room, and continuewatching the show in another location, such as a bedroom. Consumersexpect to be able to begin watching the show from the same point fromthe other location. In a commercial setting, such as a sports bar, aproprietor controls the content displayed on multiple televisions sothat the same or different content is displayed on numerous screens.Networking televisions through their set top boxes provides these andother features to residential and commercial consumers.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an exemplary system for synchronizing playback of amedia content instance across multiple devices.

FIG. 2 is a block diagram of an exemplary set top box for synchronizingplayback of a media content instance.

FIG. 3 is a flowchart of an exemplary process that may be used tosynchronize playback of a media content instance across multipledevices.

DETAILED DESCRIPTION

Synchronizing playback of media content across different rooms can bechallenging. Multiple televisions playing the same video and audio inclose proximity out of sync can be a source of annoyance andfrustration. One way to relieve such frustration is to synchronize thedecoding and playback of the media content instance. An exemplary systemfor synchronizing decoding and playback of a media content instanceincludes a server configured to receive a data stream and at least oneclient device configured to receive the data stream from the server. Theclient devices and the server each independently decode the data streambefore outputting the media content instance to a display device forpresentation to a viewer. This independent processing may cause delaysrelative to the playback of the media content instance on the server,the client devices, or both. Accordingly, the server is configured tosynchronize playback of a media content instance across these devices.The synchronization is performed in accordance with a timestamp, such asa presentation timestamp, associated with the data stream. An exemplarymethod includes receiving the data stream at the server, transmittingthe data stream to at least one client device, processing the datastream, and synchronizing playback of a media content instance inaccordance with a timestamp associated with the data stream.

The system shown in the FIGS. may take many different forms and includemultiple and/or alternate components and facilities. The exemplarycomponents illustrated are not intended to be limiting. Indeed,additional or alternative components and/or implementations may be used.

As illustrated in FIG. 1, the system 100 includes a content server 105,at least one client device 110 (only two are shown for purposes ofsimplicity), and a local server 115. The local server 115 is incommunication with the content server 105 over a service providernetwork 120 and in communication with the client devices 110 over alocal network 125. Examples of the local server 115, client device 110,or both, may include a set top box, a game console, a media contentstreaming device, or the like.

The content server 105 may be configured to provide media content to thelocal server 115. The content server 105 may access one or more contentdatabases to retrieve a media content instance and transmit the mediacontent instance, as a data stream, to the local server 115. The datastream may include audio, video, or both. Examples of a media contentinstance, therefore, may include a television show, movie, video, game,song, or the like. The content server 105 may be configured to transmitthe data stream to the local server 115 via the content providernetwork.

Each client device 110 may be configured to decode data streamscontaining a media content instance. Decoding the data stream mayinclude converting the data stream into a format that can be presentedon a display device 130, such as a television, computer monitor, or thelike. For instance, as discussed above, the data stream may includeaudio, video, or both, of the media content instance. Thus, decoding thedata stream may include a digital signal processing technique thatmanipulates the audio and video components of the data stream so thatthe media content instance may be presented by the display device 130.As discussed in greater detail below, each client device 110 may receivethe data stream from the local server 115. Moreover, each client device110 may be configured to independently decode the data stream relativeto both the local server 115 and other client devices 110. Each clientdevice 110 may be configured to receive the data stream through a wiredcommunication interface 135, a wireless communication interface 135, orboth. The data stream may be received at each client device 110 throughthe local network 125.

The local server 115 may be configured to receive the data stream fromthe content server 105 over the service provider network 120. The localserver 115 may be configured to process the data stream independently ofthe processing performed by one or more of the client devices 110. Asdiscussed above, processing the data stream may include, e.g., decodingthe data stream to a format that may allow the media content instance tobe played on, e.g., a television or other display device 130. In somepossible implementations, the local server 115 may be configured toreceive the data stream over the content provider network through awireless communication interface 135, a wired communication interface135, or both. Moreover, the local server 115 may be configured totransmit the data stream to one or more client devices 110 over thelocal network 125. The local server 115 may communicate over the localnetwork 125 through a wireless or wired communication interface 135.Because the client devices 110 may process the data stream independentlyfrom the local server 115, the local server 115 may transmit theunprocessed data stream to one or more client devices 110 over the localnetwork 125.

The local server 115 may be further configured to synchronize decoding,playback, or both, of the media content instance across multipledevices, such as the client devices 110. In one possible implementation,the local server 115 may synchronize decoding and playback of the mediacontent instance in accordance with a timestamp associated with the datastream. The timestamp may, in some instances, be embedded in the datastream as metadata. The timestamp may indicate the time at which certainportions of the media content instance are to be played. For instance,the timestamp may include a presentation timestamp (PTS) otherwise usedto synchronize audio and video portions of the media content instancewith one another. That is, the presentation timestamp may be furtherused to synchronize the decoding and playback of the media contentinstance across multiple devices. The local server 115 may be configuredto determine, based on the presentation timestamp, whether the playbackof the media content instance by one of the client devices 110 lags theplayback of the media content instance by the local server 115 oranother of the client devices 110. Likewise, the local server 115 may beconfigured to determine, based on the presentation timestamp, whetherthe playback of the media content instance by the local server 115 lagsthe playback of the media content instance by one or more of the clientdevices 110. If any lag is detected, the local server 115 may beconfigured to slow its decoding and playback of the media contentinstance, or in some instances, command one or more client devices 110to speed up their decoding of the media content instance until theplayback of the media content instance is synchronized across alldevices, including the client devices 110 and the local server 115,playing the media content instance. Alternatively or in addition, theserver 115 could speed up its decoding of the media content instance ifthe playback by the server 115 lags relative to the client devices 110.

FIG. 2 is a block diagram of an exemplary local server 115, clientdevice 110, or both. While the discussion of FIG. 2 is generally in thecontext of the local server 115, the diagram shown in FIG. 2 may applyto one or more client devices 110 or any other device that processes thedata stream and outputs the processed data stream to a display device130 to present the media content instance. As shown, the local server115 includes a communication interface 135, a decoder 140, a displayinterface 145, and a processing device 150.

The communication interface 135 may be configured to facilitate wiredand/or wireless communication over the local network 125, the contentprovider network, or both, using any number of communication protocolssuch as WiFi, Bluetooth®, Ethernet, or the like. The communicationprotocols implemented by the communication interface 135 may includewired communication protocols, wireless communication protocols, orboth. The communication interface 135, therefore, may be configured toreceive the data stream from the content server 105 over the contentprovider network as well as transmit the data stream to the clientdevices 110 over the local area network.

The decoder 140 may be configured to process the data stream receivedvia the communication interface 135. The data stream may be encoded bythe content server 105 or another device before the data stream istransmitted over the content provider network. The decoder 140 may beconfigured to decode the encoded data stream. The decoder 140 may beconfigured to decode the data stream in accordance with a format thatmay be output to the display device 130.

The display interface 145 may be configured to connect the local server115 to the display device 130. The display interface 145 may comply withany number of analog or digital display protocols. Examples of displayprotocols may include a High-Definition Multimedia Interface (HDMI), aDigital Visual Interface (DVI), a Composite video interface, an S-Videointerface, a Component video interface, etc.

The processing device 150 may be configured to facilitate thetransmission of the data stream to one or more client devices 110, andin some instances, synchronize the decoding and playback of the mediacontent instance across multiple devices, including multiple clientdevices 110 and the local server 115. For instance, the processingdevice 150 may synchronize decoding and playback of the media contentinstance in accordance with a timestamp associated with the data stream.As discussed above, the timestamp may be embedded in the data stream asmetadata, such as the presentation timestamp, indicating the time atwhich certain portions of the media content instance are to be played.The processing device 150 may be configured to determine, based on thepresentation timestamp, whether the playback of the media contentinstance by one of the client devices 110 lags the playback of the mediacontent instance by the local server 115 or another of the clientdevices 110. Likewise, the processing device 150 may be configured todetermine, based on the presentation timestamp, whether the playback ofthe media content instance by the local server 115 lags the playback ofthe media content instance by one or more of the client devices 110. Ifany lag is detected, the processing device 150 may be configured to slowits playback of the media content instance, or in some instances,command, via the communication interface 135, one or more client devices110 to slow their playback of the media content instance until theplayback of the media content instance is synchronized across alldevices, including the client devices 110 and the local server 115,playing the media content instance. Alternatively or in addition, theprocessing device 150 could speed up the decoding of the media contentinstance if the playback by the server 115 lags relative to the clientdevices 110 or command the client devices 110 to speed up their decodingof the media content instance if the playback on the client devices 110lags relative to the server 115.

FIG. 3 is a flowchart of an exemplary process 300 that may beimplemented by one or more components of the system 100 of FIG. 1.

At block 305, the local server 115 may receive the data stream from thecontent server 105 over, e.g., the service provider network 120. In somepossible implementations, the local server 115 may query the contentserver 105 for a particular media content instance, and the contentserver 105 may transmit the data stream associated with the requestedmedia content instance in response to the query from the local server115. The data stream may be received through a wired or wirelesscommunication protocol.

At block 310, the local server 115 may transmit the data stream to oneor more client devices 110. The data stream may be transmitted to theclient devices 110 over, e.g., the local network 125. The data streammay be transmitted to one or more client devices 110 through a wiredcommunication protocol, a wireless communication protocol, or both.

At block 315, the local server 115 and any client devices 110 thatreceived the data stream at block 310 may independently process the datastream. As discussed above, processing the data stream may includeconverting the data stream into a format that can be presented on adisplay device 130, such as a television, computer monitor, or the like.Specifically, in one possible approach, the data stream may be encodedby the content server 105 or another device before the data stream istransmitted to the local server 115 over the content provider network.Thus, processing the data stream may include decoding the encodedversion of the data stream. The data stream may be decoded in accordancewith a format that may be output to and presented on the display device130.

At block 320, the local server 115 may synchronize decoding, playback,or both of the media content instance across the local server 115 andone or more of the client devices 110 that are simultaneously presentingthe same media content instance on multiple display devices 135. Thelocal server 115 may synchronize decoding and playback of the mediacontent instance according to a timestamp associated with the datastream. An example timestamp may include the presentation timestamp,which may also be used for synchronizing the audio and video componentsof the media content instance. The local server 115 may determinewhether the playback of the media content instance by one of the clientdevices 110 or the local server 115 lags the playback of the mediacontent instance by any other device, including other client devices 110or the local server 115. If a lag is discovered, the local server 115may delay its own playback of the media content instance untilsynchronized, as indicated by the presentation timestamp, with one ormore of the client devices 110. Alternatively or in addition, the localserver 115 may command one or more of the client devices 110 to delayplayback until synchronized, again as indicated by the presentationtimestamp. Alternatively or in addition, the server 115 may speed up thedecoding of the media content instance if the playback by the server 115lags relative to the client devices 110 or command the client devices110 to speed up their decoding of the media content instance if theplayback on the client devices 110 lags relative to the server 115.

In general, the computing systems and/or devices may employ any of anumber of computer operating systems, including, but by no means limitedto, versions and/or varieties of the Microsoft Windows® operatingsystem, the Unix operating system (e.g., the Solaris® operating systemdistributed by Oracle Corporation of Redwood Shores, Calif.), the AIXUNIX operating system distributed by International Business Machines ofArmonk, N.Y., the Linux operating system, the Mac OS X and iOS operatingsystems distributed by Apple Inc. of Cupertino, Calif., the BlackBerryOS distributed by Research In Motion of Waterloo, Canada, and theAndroid operating system developed by the Open Handset Alliance.Examples of computing devices include, without limitation, a computerworkstation, a server, a desktop, notebook, laptop, or handheldcomputer, or some other computing system and/or device.

Computing devices generally include computer-executable instructions,where the instructions may be executable by one or more computingdevices such as those listed above. Computer-executable instructions maybe compiled or interpreted from computer programs created using avariety of programming languages and/or technologies, including, withoutlimitation, and either alone or in combination, Java™, C, C++, VisualBasic, Java Script, Perl, etc. In general, a processor (e.g., amicroprocessor) receives instructions, e.g., from a memory, acomputer-readable medium, etc., and executes these instructions, therebyperforming one or more processes, including one or more of the processesdescribed herein. Such instructions and other data may be stored andtransmitted using a variety of computer-readable media.

A computer-readable medium (also referred to as a processor-readablemedium) includes any non-transitory (e.g., tangible) medium thatparticipates in providing data (e.g., instructions) that may be read bya computer (e.g., by a processor of a computer). Such a medium may takemany forms, including, but not limited to, non-volatile media andvolatile media. Non-volatile media may include, for example, optical ormagnetic disks and other persistent memory. Volatile media may include,for example, dynamic random access memory (DRAM), which typicallyconstitutes a main memory. Such instructions may be transmitted by oneor more transmission media, including coaxial cables, copper wire andfiber optics, including the wires that comprise a system bus coupled toa processor of a computer. Common forms of computer-readable mediainclude, for example, a floppy disk, a flexible disk, hard disk,magnetic tape, any other magnetic medium, a CD-ROM, DVD, any otheroptical medium, punch cards, paper tape, any other physical medium withpatterns of holes, a RAM, a PROM, an EPROM, a FLASH-EEPROM, any othermemory chip or cartridge, or any other medium from which a computer canread.

Databases, data repositories or other data stores described herein mayinclude various kinds of mechanisms for storing, accessing, andretrieving various kinds of data, including a hierarchical database, aset of files in a file system, an application database in a proprietaryformat, a relational database management system (RDBMS), etc. Each suchdata store is generally included within a computing device employing acomputer operating system such as one of those mentioned above, and areaccessed via a network in any one or more of a variety of manners. Afile system may be accessible from a computer operating system, and mayinclude files stored in various formats. An RDBMS generally employs theStructured Query Language (SQL) in addition to a language for creating,storing, editing, and executing stored procedures, such as the PL/SQLlanguage mentioned above.

In some examples, system elements may be implemented ascomputer-readable instructions (e.g., software) on one or more computingdevices (e.g., servers, personal computers, etc.), stored on computerreadable media associated therewith (e.g., disks, memories, etc.). Acomputer program product may comprise such instructions stored oncomputer readable media for carrying out the functions described herein.

With regard to the processes, systems, methods, heuristics, etc.described herein, it should be understood that, although the steps ofsuch processes, etc. have been described as occurring according to acertain ordered sequence, such processes could be practiced with thedescribed steps performed in an order other than the order describedherein. It further should be understood that certain steps could beperformed simultaneously, that other steps could be added, or thatcertain steps described herein could be omitted. In other words, thedescriptions of processes herein are provided for the purpose ofillustrating certain embodiments, and should in no way be construed soas to limit the claims.

Accordingly, it is to be understood that the above description isintended to be illustrative and not restrictive. Many embodiments andapplications other than the examples provided would be apparent uponreading the above description. The scope should be determined, not withreference to the above description, but should instead be determinedwith reference to the appended claims, along with the full scope ofequivalents to which such claims are entitled. It is anticipated andintended that future developments will occur in the technologiesdiscussed herein, and that the disclosed systems and methods will beincorporated into such future embodiments. In sum, it should beunderstood that the application is capable of modification andvariation.

All terms used in the claims are intended to be given their broadestreasonable constructions and their ordinary meanings as understood bythose knowledgeable in the technologies described herein unless anexplicit indication to the contrary is made herein. In particular, useof the singular articles such as “a,” “the,” “said,” etc. should be readto recite one or more of the indicated elements unless a claim recitesan explicit limitation to the contrary.

The Abstract of the Disclosure is provided to allow the reader toquickly ascertain the nature of the technical disclosure. It issubmitted with the understanding that it will not be used to interpretor limit the scope or meaning of the claims. In addition, in theforegoing Detailed Description, it can be seen that various features aregrouped together in various embodiments for the purpose of streamliningthe disclosure. This method of disclosure is not to be interpreted asreflecting an intention that the claimed embodiments require morefeatures than are expressly recited in each claim. Rather, as thefollowing claims reflect, inventive subject matter lies in less than allfeatures of a single disclosed embodiment. Thus the following claims arehereby incorporated into the Detailed Description, with each claimstanding on its own as a separately claimed subject matter.

1. A system comprising: a server configured to receive a data stream;and a client device configured to receive the data stream from theserver, wherein the server is configured to synchronize playback of amedia content instance by the server and the client device in accordancewith a timestamp associated with the data stream.
 2. The system of claim1, wherein the data stream includes the media content instance.
 3. Thesystem of claim 1, wherein the server and the client device areconfigured to independently process the data stream.
 4. The system ofclaim 1, wherein the server is configured to determine whether theplayback of the media content instance by the client device lags theplayback of the media content instance by the server.
 5. The system ofclaim 1, wherein the client device is configured to determine whetherthe playback of the media content instance by the server lags theplayback of the media content instance by the client device.
 6. Thesystem of claim 1, wherein the server is configured to receive the datastream through at least one of a wired communication interface and awireless communication interface.
 7. The system of claim 1, wherein theclient device is configured to receive the data stream through at leastone of a wireless communication interface and a wired communicationinterface.
 8. A system comprising: a server configured to output a datastream; a first client device in communication with the server andconfigured to process the data stream; and a second client device incommunication with the server and configured to process the data stream,wherein the server is configured to synchronize playback of a mediacontent instance by the first client device and the second client devicein accordance with a timestamp associated with the data stream.
 9. Thesystem of claim 8, wherein the data stream includes the media contentinstance.
 10. The system of claim 8, wherein the first client device andthe second client device are configured to independently decode the datastream.
 11. The system of claim 8, wherein the server is configured todetermine whether the playback of the media content instance by thefirst client device lags relative to the playback of the media contentinstance by the second client device.
 12. The system of claim 8, whereinthe server is configured to receive the data stream through a wiredcommunication interface.
 13. The system of claim 8, wherein the serveris configured to receive the second data stream through a wirelesscommunication interface.
 14. The system of claim 8, wherein the serveris configured to transmit the data stream through at least one of awired and wireless communication interface.
 15. The system of claim 8,wherein at least one of the first client device and the second clientdevice is configured to receive the data stream through at least one ofa wired and wireless communication interface.
 16. A method comprising:receiving, at a server, a data stream including a media contentinstance; transmitting the data stream to at least one client device;processing the data stream; and synchronizing playback of a mediacontent instance in accordance with a timestamp associated with the datastream.
 17. The method of claim 16, wherein processing the data streamincudes independently decoding the data stream by the server and the atleast one client device.
 18. The method of claim 16, whereinsynchronizing playback of the media content instance includesdetermining whether the playback of the media content instance by the atleast one client device lags relative to the playback of the mediacontent instance by the server.
 19. The method of claim 16, wherein thedata stream is received by the server through at least one of a wiredand wireless communication interface.
 20. The method of claim 16,wherein the data stream is received by the at least client devicethrough at least one of a wired and wireless communication interface.